1. Field of the Invention
The present invention relates to a visual telephone apparatus and a data compression multiplexing method therefor as well as a recording medium on which a data compression multiplexing control program is recorded, and more particularly to a method for compression production of image or speech data for a visual telephone apparatus and for multiplexing of compression image data, compression speech data and user data.
2. Description of the Related Art
Conventionally, a visual telephone apparatus of the type mentioned (also called television telephone, picturephone and so forth) is used in order to communicate images, speech and user data such as an electronic file over a common public network as recited, for example, in the TTC standards (draft) JT-H324 of a material for the 23rd standardization conference by the Telecommunication Technology Committee.
Referring to FIG. 9, there is shown in block diagram an exemplary one of conventional visual telephone apparatus. A video I/O (input/output) apparatus 31 images a person who is in front of the visual telephone apparatus 30, and outputs an image signal obtained by the imaging. A video encoder 32 compresses the image signal inputted thereto from the video I/O apparatus 31 and outputs a resulting compressed image signal.
An audio I/O apparatus 33 collects speech of a person who is in front of the visual telephone apparatus 30, and outputs the collected speech signal. An audio encoder 34 compresses the speech signal outputted from the audio I/O apparatus 33 and outputs a resulting compressed speech signal. A reception path delay section 35 produces, from a difference between the processing time for image compression and the processing time for speech compression, a delay for preventing a time difference from being produced between outputting of speech and displaying of an image.
A user data application 36 is a txc3xa9lxc3xa9matique apparatus like a computer which outputs user data and may be, for example, such a shared white board as shown in FIG. 8. Referring to FIG. 8, in the shared white board, for example, if an expression xe2x80x9cone, to, three, fourxe2x80x9d is inputted to a white board 22 of a terminal B using a pen 22a, then the expression xe2x80x9cone, to, three, fourxe2x80x9d of the white board 22 of the terminal B is displayed on a white board 21 of another terminal A, and that the term xe2x80x9ctoxe2x80x9d is wrong can be pointed out from the terminal B by placing an underline below the word xe2x80x9ctoxe2x80x9d using a pen 21a of the terminal A. In this manner, a white board is shared by the terminals A and B.
Referring back to FIG. 9, a data protocol section 37 performs framing of user data outputted from the user data application 36. A system controller 38 develops an instruction of negotiation with the other party visual telephone apparatus (not shown) and an instruction for modem control. A control protocol section 39 converts the instruction of negotiation of the control protocol section 39 to predetermined protocol data.
An SRP (Simple Retransmission Protocol) or LAPM (Link Access Protocol for Modems) (SRP/LAPM) procedure section 40 performs a retransmission procedure for protocol data outputted from the control protocol section 39. A multiplexer/demultiplexer 41 multiplexes compressed image data outputted from the video encoder 32, compressed speech data outputted from the reception path delay section 35, user data outputted from the data protocol section 37 and protocol data outputted from the SRP/LAPM procedure section 40, and outputs resulting multiplexed data.
A modem 42 outputs the multiplexed data outputted from the multiplexer/demultiplexer 41 to a GSTN (General Switched Telephone Network) network 44. A modem controller 43 controls the modem 42 in accordance with a modem control instruction outputted from the system controller 38. An MCU (Multipoint Control Unit) 45 performs control when visual telephone communication is performed at multiple points.
In operation, referring to FIG. 9, as an initial condition, the system controller 38 issues a modem control instruction to the modem controller 43, and the modem 42 which receives the modem control instruction performs communication originating processing to the other party visual telephone apparatus or MCU 45.
If a communication channel is established through the GSTN network 44, then the system controller 38 issues an instruction of negotiation with the other party terminal to the control protocol section 39. The control protocol section 39 produces protocol data from the negotiation instruction.
The protocol data are subject to a retransmission procedure by the SRP/LAPM procedure section 40 and then multiplexing processing by the multiplexer/demultiplexer 41, whereafter they are transferred to the other party visual telephone apparatus over the GSTN network 44 by the modem 42. By this operation, matching with regard to the capacity and so forth is established between the visual telephone apparatus 30 and the other party visual telephone apparatus. After the matching operation is completed, a communication condition is entered.
In the communication condition, the video encoder 32 performs compression processing for an image imaged by the video I/O apparatus 31 and outputs a resulting compressed image signal. Simultaneously, the audio encoder 34 performs compression processing for speech collected by the audio I/O apparatus 33 and outputs resulting compressed speech data.
On the transmission side, the reception path delay section 35 outputs the compressed speech data as they are. User data outputted from the user data application 36 such as a shared white board are subject to framing processing by the data protocol section 37.
Further, an instruction for mode setting or the like outputted from the system controller 38 in accordance with a situation is converted into protocol data by the control protocol section 39 and then subject to a retransmission procedure by the SRP/LAPM procedure section 40 so that control protocol data are outputted. The multiplexer/demultiplexer 41 here performs multiplexing processing of media for the compressed image data outputted from the video encoder 32, the compressed speech data outputted from the reception path delay section 35, the framed user data outputted from the data protocol section 37 and the control protocol data outputted from the SRP/LAPM procedure section 40 in accordance with a situation, and outputs resulting multiplexed data.
The multiplexed data are transferred to the other party visual telephone apparatus over the GSTN network 44 from the modem 42. The multiplexed data received at the modem 42 from the visual telephone apparatus of the originating party are demultiplexed into compressed image data, compressed speech data, user data and control protocol data, which are outputted to the video encoder 32, reception path delay section 35, data protocol section 37 and SRP/LAPM procedure section 40, respectively. The video encoder 32 performs decompression processing for the compressed image data and outputs a resulting image signal. The image signal is displayed by the video I/O apparatus 31.
The reception path delay section 35 performs delaying processing for compensating for a synchronism displacement between the image and the speech which is produced from a difference between the processing time for coding of the image data and the processing time for coding of the speech data. The compressed speech data outputted from the reception path delay section 35 with the synchronism displacement thereof compensated for are subject to decompression processing by the audio encoder 34 so that they are converted into a speech signal. The speech signal is outputted as speech by the audio I/O apparatus 33.
The data protocol section 37 performs deframing processing for the framed user data received from the multiplexer/demultiplexer 41 and outputs resulting user data. The user data are processed by the user data application 36. Meanwhile, the control protocol data outputted from the multiplexer/demultiplexer 41 are subject to processing of a retransmission procedure by the SRP/LAPM procedure section 40 and are then converted into an instruction for mode setting or the like by the control protocol section 39. The instruction is processed by the system controller 38.
In the conventional visual telephone apparatus described above, normally an image and speech are communicated with a visual telephone apparatus of the other party, and bands for the image and the speech are secured preferentially by the multiplexer/demultiplexer. Thereupon, if the user data application operates to output a large amount of user data, then if, upon multiplexing, the bands for image and speech data are secured preferentially in a usual manner, then the band for user data cannot be secured. Consequently, the throughput of the user data becomes low, and as a result, the speed of response of the user data application becomes low.
Further, usually the image compression section determines the compression ratio for image compression in accordance with a parameter set in advance by an operation of a user before visual telephone communication is started. In this instance, although the user data application outputs a large amount of user data, if the multiplexer/demultiplexer sets the band for image data to a comparatively narrow band to raise the throughput of user data, then the throughput of the compressed image data is deteriorated, and this results in such a bad effect as degradation of the frame rate of regenerated images.
Also, usually the speech compression section determines the compression ratio for speech compression in accordance with a parameter set in advance by an operation of the user before visual telephone communication is started. In this instance, although the user data application outputs a large amount of user data, if the multiplexer/demultiplexer sets the band for speech data to a comparatively narrow band to raise the throughput of user data, then the throughput of compressed speech data is deteriorated, and this results in such a bad effect as interruption of regenerated sound.
It is an object of the present invention to provide a visual telephone apparatus and a data compression multiplexing method therefor as well as a recording medium having a data compression multiplexing control program recorded thereon wherein a high speed of response of a user data application can be secured.
It is another object of the present invention to provide a visual telephone apparatus and a data compression multiplexing method therefor as well as a recording medium having a data compression multiplexing control program recorded thereon wherein a band for user data in multiplexing can be set freely so that the band for user data can be secured preferentially and a comparatively high speed of response of a user data application can be secured.
It is a further object of the present invention to provide a visual telephone apparatus and a data compression multiplexing method therefor as well as a recording medium having a data compression multiplexing control program recorded thereon wherein the compression ratio for image compression can be set freely so that a comparatively narrow band can be set to image data in multiplexing while the band for user data can be secured preferentially and a comparatively high speed of response of a user data application can be secured.
It is a still further object of the present invention to provide a visual telephone apparatus and a data compression multiplexing method therefor as well as a recording medium having a data compression multiplexing control program recorded thereon wherein the compression ratio for speech compression can be set freely so that a comparatively narrow band can be allocated to speech data in multiplexing while the band for user data can be secured preferentially and a comparatively high speed of response of a user data application can be secured.
In order to attain the objects described above, according to an aspect of the present invention, there is provided a visual telephone apparatus, comprising speech compression means for compressing speech to be sent out to an object apparatus of communication, image compression means for compressing an image to be sent out to the object apparatus of communication, a user data application for producing data including at least character information to be sent out to the object apparatus of communication, inputting means for inputting the data to the user data application, multiplexing means for multiplexing the speech compressed by the speech compression means, the image compressed by the image compression means and the data produced by the user data application and sending out resulting multiplexed data to the object apparatus of communication, first detection means for detecting whether or not the user data application is operating, second detection means for detecting, when it is detected by the first detection means that the user data application is operating, an amount of input data inputted from the inputting means per unit time set in advance, and means for producing band control information for setting a band to be allocated to the data produced by the user data application in response to a result of the detection of the second detection means and outputting the band control information to the multiplexing means.
In the visual telephone apparatus, the ratio of bands to be allocated to user data and compressed image and speech data in multiplexing is variably set depending upon whether or not the user data application is operating and based on the input data amount of the inputting means such as a keyboard and/or a mouse per unit time.
More particularly, it is detected whether or not the user data application is operating, and the input data amount inputted from the inputting means per unit time is detected. Then, in response to the detected input data amount, the band control information for variably setting a band to be allocated to the data produced by the user data application is produced and outputted to the multiplexing means. The multiplexing means thus variably sets the ratio of bands to be allocated to the data and the compressed image and compressed speech in response to the band control information.
Since the band to be allocated to the user data in accordance with an operation situation of the user data application depending upon whether or not the user data is operating and in response to the input data amount inputted from the inputting means per unit time, the band for the user data upon multiplexing can be set freely, and consequently, the band for the user data can be set preferentially and the speed of response of the user data application can be raised.
According to another aspect of the present invention, there is provided a visual telephone apparatus, comprising speech compression means for compressing speech to be sent out to an object apparatus of communication, image compression means for compressing an image to be sent out to the object apparatus of communication, a user data application for producing data including at least character information to be sent out to the object apparatus of communication, inputting means for inputting the data to the user data application, multiplexing means for multiplexing the speech compressed by the speech compression means, the image compressed by the image compression means and the data produced by the user data application and sending out resulting multiplexed data to the object apparatus of communication, first detection means for detecting whether or not the user data application is operating, second detection means for detecting, when it is detected by the first detection means that the user data application is operating, an amount of input data inputted from the inputting means per unit time set in advance, and means for producing speech compression control information for setting a speech compression ratio of the speech compression means in response to a result of the detection of the second detection means and outputting the speech compression control information to the speech compression means.
In the visual telephone apparatus, the speech compression ratio is variably set depending upon whether or not the user data is operating and based on the input data amount of the inputting means such as a keyboard and/or a mouse per unit time. More particularly, it is detected whether or not the user data application is operating, and the input data amount inputted from the inputting means per unit time is detected. Then, in response to the detected input data amount, the control information for variably setting the speech compression ratio is produced and outputted to the speech compression section. The speech compression section thus variably sets the speech compression ratio in accordance with the control information.
Since the speech compression ratio when an inputted speech signal is to be compressed is determined in accordance with an operation situation of the user data application particularly depending upon whether or not the user data is operating and in response to the input data amount inputted from the inputting means per unit time, while the user application is operating, the band for the transmission compressed speech data upon multiplexing can be set low and the band for the user data can be set preferentially. Consequently, the speed of response of the user data application can be raised while preventing interruption of reproduction speech of the other party visual telephone apparatus.
According to a further aspect of the present invention, there is provided a visual telephone apparatus, comprising speech compression means for compressing speech to be sent out to an object apparatus of communication, image compression means for compressing an image to be sent out to the object apparatus of communication, a user data application for producing data including at least character information to be sent out to the object apparatus of communication, inputting means for inputting the data to the user data application, multiplexing means for multiplexing the speech compressed by the speech compression means, the image compressed by the image compression means and the data produced by the user data application and sending out resulting multiplexed data to the object apparatus of communication, first detection means for detecting whether or not the user data application is operating, second detection means for detecting, when it is detected by the first detection means that the user data application is operating, an amount of input data inputted from the inputting means per unit time set in advance, and means for producing image compression control information for setting an image compression ratio of the image compression means in response to a result of the detection of the second detection means and outputting the image compression control information.
In the visual telephone apparatus, the image compression ratio is variably set depending upon whether or not the user data is operating and based on the input data amount of the inputting means such as a keyboard and/or a mouse per unit time. More particularly, it is detected whether or not the user data application is operating, and the input data amount inputted from the inputting means per unit time is detected. Then, in response to the detected input data amount, the control information for variably setting the image compression ratio is produced and outputted to the image compression section. The image compression section thus variably sets the image compression ratio in accordance with the control information.
Since the image compression ratio when an inputted image signal is to be compressed is determined in accordance with an operation situation of the user data application particularly depending upon whether or not the user data is operating and in response to the input data amount inputted from the inputting means per unit time, while the user application is operating, the band for the transmission compressed image data upon multiplexing can be set low and the band for the user data can be set preferentially. Consequently, the speed of response of the user data application can be raised while preventing a drop of the frame rate of reproduction images of the other party visual telephone apparatus.
According to a still further aspect of the present invention, there is provided a data compression multiplexing method for a visual telephone apparatus which includes speech compression means for compressing speech to be sent out to an object apparatus of communication, image compression means for compressing an image to be sent out to the object apparatus of communication, a user data application for producing data including at least character information to be sent out to the object apparatus of communication, inputting means for inputting the data to the user data application, and multiplexing means for multiplexing the speech compressed by the speech compression means, the image compressed by the image compression means and the data produced by the user data application and sending out resulting multiplexed data to the object apparatus of communication, the data compression multiplexing method comprising the steps of detecting whether or not the user data application is operating, detecting, when it is detected that the user data application is operating, an amount of input data inputted from the inputting means per unit time set in advance, and producing band control information for setting a band to be allocated to the data produced by the user data application in response to a result of the detection and outputting the band control information to the multiplexing means.
According to a yet further aspect of the present invention, there is provided a data compression multiplexing method for a visual telephone apparatus which includes speech compression means for compressing speech to be sent out to an object apparatus of communication, image compression means for compressing an image to be sent out to the object apparatus of communication, a user data application for producing data including at least character information to be sent out to the object apparatus of communication, inputting means for inputting the data to the user data application, and multiplexing means for multiplexing the speech compressed by the speech compression means, the image compressed by the image compression means and the data produced by the user data application and sending out resulting multiplexed data to the object apparatus of communication, the data compression multiplexing method comprising the steps of detecting whether or not the user data application is operating, detecting, when it is detected that the user data application is operating, an amount of input data inputted from the inputting means per unit time set in advance, and producing speech compression control information for setting a speech compression ratio of the speech compression means in response to a result of the detection and outputting the speech compression control information to the speech compression means.
According to a yet further aspect of the present invention, there is provided a data compression multiplexing method for a visual telephone apparatus which includes speech compression means for compressing speech to be sent out to an object apparatus of communication, image compression means for compressing an image to be sent out to the object apparatus of communication, a user data application for producing data including at least character information to be sent out to the object apparatus of communication, inputting means for inputting the data to the user data application, and multiplexing means for multiplexing the speech compressed by the speech compression means, the image compressed by the image compression means and the data produced by the user data application and sending out resulting multiplexed data to the object apparatus of communication, the data compression multiplexing method comprising the steps of detecting whether or not the user data application is operating, detecting, when it is detected that the user data application is operating, an amount of input data inputted from the inputting means per unit time set in advance, and producing image compression control information for setting an image compression ratio of the image compression means in response to a result of the detection and outputting the image compression control information.
According to a yet further aspect of the present invention, there is provided a recording medium having recorded thereon a data compression multiplexing control program for a visual telephone apparatus which includes speech compression means for compressing speech to be sent out to an object apparatus of communication, image compression means for compressing an image to be sent out to the object apparatus of communication, a user data application for producing data including at least character information to be sent out to the object apparatus of communication, inputting means for inputting the data to the user data application, and multiplexing means for multiplexing the speech compressed by the speech compression means, the image compressed by the image compression means and the data produced by the user data application and sending out resulting multiplexed data to the object apparatus of communication, the data compression multiplexing control program causing control means for the speech compression means, the image compression means and the multiplexing means to detect whether or not the user data application is operating, detect, when it is detected that the user data application is operating, an amount of input data inputted from the inputting means per unit time set in advance, and produce band control information for setting a band to be allocated to the data produced by the user data application in response to a result of the detection and output the band control information to the multiplexing means.
According to a yet further aspect of the present invention, there is provided a recording medium having recorded thereon a data compression multiplexing control program for a visual telephone apparatus which includes speech compression means for compressing speech to be sent out to an object apparatus of communication, image compression means for compressing an image to be sent out to the object apparatus of communication, a user data application for producing data including at least character information to be sent out to the object apparatus of communication, inputting means for inputting the data to the user data application, and multiplexing means for multiplexing the speech compressed by the speech compression means, the image compressed by the image compression means and the data produced by the user data application and sending out resulting multiplexed data to the object apparatus of communication, the data compression multiplexing control program causing control means for the speech compression means, the image compression means and the multiplexing means to detect whether or not the user data application is operating, detect, when it is detected that the user data application is operating, an amount of input data inputted from the inputting means per unit time set in advance, and produce speech compression control information for setting a speech compression ratio of the speech compression means in response to a result of the detection and outputting the speech compression control information to the speech compression means.
According to a yet further aspect of the present invention, there is provided a recording medium having recorded thereon a data compression multiplexing control program for a visual telephone apparatus which includes speech compression means for compressing speech to be sent out to an object apparatus of communication, image compression means for compressing an image to be sent out to the object apparatus of communication, a user data application for producing data including at least character information to be sent out to the object apparatus of communication, inputting means for inputting the data to the user data application, and multiplexing means for multiplexing the speech compressed by the speech compression means, the image compressed by the image compression means and the data produced by the user data application and sending out resulting multiplexed data to the object apparatus of communication, the data compression multiplexing control program causing control means for the speech compression means, the image compression means and the multiplexing means to detect whether or not the user data application is operating, detect, when it is detected that the user data application is operating, an amount of input data inputted from the inputting means per unit time set in advance, and produce image compression control information for setting an image compression ratio of the image compression means in response to a result of the detection and outputting the image compression control information.
The above and other objects, features and advantages of the present invention will become apparent from the following description and the appended claims, taken in conjunction with the accompanying drawings in which like parts or elements are denoted by like reference symbols.